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ABSTRACT 



This paper reports on research to study the benefit of a 
punctual self -directed learning experience with a hypermedia in the 
appropriation of a foreign language. Twenty-eight educational sciences 
students took a 55 item pretest on English notions. The students then engaged 
in a self -directed learning session using the CD-ROM, "L' anglais 
d ' au j ourd ' hui en 90 lecons," (a "Studio Multimedia" product) and later took a 
post-test consisting of the same 55 questions as the pretest. Results are 
discussed in the following areas: (1) the concept of "spectrum of 

performances," i.e., the distribution of a student's answers according to 
their type along a spectrum from incorrect with a high degree of confidence 
to correct with a high degree of confidence; (2) the notion of "ideal 
spectral evolution"; (3) quantification of the four hemispectrum shapes; and 
(4) categorization of individual cases as outnorming students, ideal 
students, stationary students, radicalized students, weak students aware of 
their progresses, and students unaware of their progresses. Figures and 
tables illustrate: confidence degrees; states of partial knowledge; ideal 
spectral evolution; expected spectral evolution; skewness coefficients; and 
one student's spectrum of performances, performances card, and realism 
graphs. Contains 22 references. (DLS ) 
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Abstract: Today we are witnessing an explosion of the learning resources on the one hand 
and on the other hand a need of continuous learning [Leclercq & Denis 95]. In this context 
autonomous learning will take an increasing part of the educational process and people will 
turn more and more towards computerised products. As educationists, we study how and 
what people learn by exploring these resources... to be able afterwards to offer them helps to 
improve their learning strategies... and outcomes. 

For one year, we have been conducting a research on learning strategies with a CD-Rom on 
English. Our research has been attempting to study the benefit of a punctual self-directed 
learning experience with a hypermedia in the appropriation of a foreign language. Our two 
main research questions are : (1) What is the evolution of each student’s “spectrum of 
performances” between the pretest and the post-test ? (2) Which learning strategy does 
he/she use and what is his/her efficiency ? 



Context 

In the current context of self-directed learning, people turn more and more towards computerised products... 
The Task force’s report on Multimedia Educational Software edited by the European Commission (1996) shows 
for example the predicted evolution of CD-Rom drives in families between 1994 and 1998 (for Europe, 2.74 
millions in 1994 and 35.96 millions in 1998). 

We start from the idea that a lot of commercial tools do exist on the market and are actually used. As 
educationists we have no control on their diffusion but we can turn the situation to best account and formulate 
recommendations concerning their use. That is why we study how and what people learn by exploring these 
computerised informational resources... to be able afterwards to offer them helps to improve their learning 
strategies... and outcomes. 



Research Object and Questions 

For one year, we have been conducting a research on learning strategies exhibited by students using a 
hypermedia on English (as a Foreign Language). Our main research question is : “ What is the benefit of a 
punctual self-directed learning experience with a hypermedia in the appropriation of a foreign language ? ”. 
More precisely, we attempt to answer the two following questions : (1) What is the evolution of each student’s 
“ spectrum of performances ” (see [Concept of “ Spectrum of Performances ”]) between the pretest and the post- 
test ? (2) Which learning strategy does he/she use and what is his/her efficiency ? 



Progress of the Experiment 
Pretest 
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Our main objective is to measure the impact of self-directed learning by exploring three English lessons in the 
CD-Rom “ L’anglais d’aujourd’hui en 90 let^ons ” (a “ Studio Multimedia ” product). Therefore, last year, 28 
students registered in the first year of the licence in Educational Sciences received a pretest containing 55 items 
(short open-ended questions) on English notions. These notions represent 55 “ opportunities to learn ” (OTL) 
precisely located in the hypermedia lessons. Students had to complete the test and accompany each of their 
answers with one of the six following “ confidence degrees 
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Figure 1: confidence degrees 

The reasons for this (asymmetric) segmentation of the probability axis have been explained elsewhere [Leclercq 
83 & 93] and are related to what is currently known of the human capacity in discriminating different levels of 
doubt (or partial knowledge). 



Self-Directed Learning Session 

At the beginning of the self-directed learning session, students received directions for navigation and got 
familiar with the possibilities of the hypermedia. Later, students explored individually three lessons of the 
hypermedia (n° 25, 35 and 40) during one hour as a maximum. Half of the students received back their pretest 
during the self-directed learning session with a short feed-back on their answers (correct answer/incorrect 
answer) ; the other half did not receive anything. This experimental design aims to compare learning strategies 
of students with “ objective learning needs ” and learning strategies of students with “ subjective learning 
needs ”. The first situation is similar to school situation ; the second one to everyday life. 

After the self-directed learning session, students were given a self-assessment questionnaire on their degree of 
appreciation of the self-directed learning session ; on their initial objectives ; on their learning strategies ; on 
their feeling to have learned something ; on their computer literacy ; on their practical experience in self- 
directed learning ... 

All the learner’s actions in using the hypermediated lessons were video-taped with the help of a “ Maxi 
Converter Pro ” system, a hardware converter that allows to display the computer screen on a television screen 
and/or to record the tracks on a video cassette. 



Post-test 

Students received the same 55 questions as in the pretest. The post-test was not given immediately after 
learning, but postponed, since students have not had the learning session at the same time and because we 
prefer to measure long-term effects of self-directed learning. 



Analysis of the Results 

Concept of “ Spectrum of Performances ” 

Too often the answer to a question is considered in a binary way : it is correct or incorrect without any concern 
for the learner’s conviction [Gilles 96]. Nevertheless, pioneers like [De Finetti 65], [Van Naerssen 66], 
[Shuford et al. 66], followed by a lot of researchers [Bruno 93], [Dirkzwager 93], [Fabre 93], [Gilles 96], 
[Hunt, 93], [Jans 95], [Jans & Leclercq 97], Leclercq 83 & 93], [Plunus 96], [Van Lenthe 93] recommend that 
researchers deal with different states of partial knoweldge to study learning. 
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The expression of doubts with the system of confidence degrees enables to take account of these different states. 
[Plunus 96] and [Leclercq, Gilles & Jans 97] defined 6 of them, represented in a “spectrum of performances” : 
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Figure 2: 6 states of partial knowledge 

The student’s “ spectrum of performances ” can be defined as the distribution of his answers according to their 
type : from incorrect answers with a high confidence degree (the worst or leftmost part of the spectrum) to 
correct answers with a high confidence degree (the best or rightmost part of the spectrum), passing through 
errors with doubt and correct answers with doubt (intermediate states). 



Notion of “ Ideal Spectral Evolution ” 

A spectral analysis for a test will be called “ static spectral analysis ” when it concerns a fixed state. A “ bi- 
static spectral analysis ” compares two states (two spectra), for example a pretest and a post-test, i.e. the global 
evolution of the performance, but not question after question (that is the object of the dynamic analysis). 

In our experiment, two spectra for each student are drawn : the first is related to his/her pretest performances ; 
the second one concerns his/her post-test performances, so that bi-static spectral analysis will be possible. 

[Fig. 3] illustrates the kind of “ couple of spectra ” (pretest and post-test) ideally expected : 




— • — Pretest 



- Post-test 



Spectral situation of answers 



Figure 3: example of “ideal spectral evolution” 

The following observations are expected : 





Incorrect Answers 


Correct answers 


PRE 


No matter what the content is, they should 
always be accompanied by a low confidence 
degree (CD) 


No special expectation. According to the test 
content (easy or difficult for the student), he/she 
will know a bit or more partially, with CD between 
0 and 5. In our example, they are distributed 
uniformly. 


POST 
(similar 
to the 
pretest) 


Their number should decrease, mainly 
answers with a high CD. The curve should 
present an escarpment more pronounced on 
the right (J-shaped curve). 


They should be more numerous and more confident. 
They should also present a J-shaped curve. 





The escarpment increase is mainly due to a 


The escarpment increase is mainly due to a 




realism increase. 


competence increase. 



Table 4: expected spectral evolution 



Quantification of the Four Hemispectrum Shapes 



The escarpment or steepness of the two “ curves ” (strictly, histograms should replace the curves ; we decided to 
join the tops of histograms in order to facilitate the comparison between pretest and post-test) can be quantified 
[Laveault & Grdgoire 97] with the asymmetry coefficient (skewness). This mathematical function characterizes 
the asymmetrical degree of a distribution according to the mean. A positive asymmetry means a distribution 
with its “tail” moved forward the right side of the graph (i-shaped curve). A negative asymmetry means a 
distribution with its “tail” moved forward the left side of the graph (j-shaped curve). 



The skewness index equation is : 



Cn - l)(n - 2) 



21 



Here are the skewness coefficients computed for the four curves of our example. If the series of intervals of our 
six confidence degrees were symetrical, the skewness of the right side pretest curve would have been 0. 





Incorrect answers (left side) 


Correct answers (right side) 


Protest 


- 0.49 


- 0.14 


Post-test 


- 2.03 


- 1.15 



Table 5: skewness coefficients 



Categorization of Individual Cases 

Our first experimentation led us to distinguish 6 types of students according to their “spectrum of 
performances” and their “performances card”, i.e. their pre- and post-test results, their (absolute and relative) 
gains, their indices and graphs of realism, their skewness indices : (1) outnorming students (overcompetent or 
undercompetent) ; (2) ideal students, aware of their (important) progresses ; (3) stationary students ; (4) 
radicalized students (at post-test) ; (5) weak students, aware of their progresses ; (6) students unaware of their 
progresses. 

Different clues can help to explain the evolution of the “spectrum of performances”. They are related to : (1) the 
subject’s learning strategy (duration of the exploration ; lessons really explored ; consulted screens ; number of 
“seized” opportunities of learning ...) ; (2) the learner him/herself (his/her initial competency ; his/her 
objectives ; his/her motivation to explore the hypermedia ; his/her computer and hypermedia literacy ; his/her 
satisfaction after the learning phase ...) ; (3) the learner’s knowledge of pretest performance (objective feedback 
or subjective impression ?). 

Detailed results are analysed in a previous article [Jans 97]. This paper presents only one example, Student 1 1, 
an “ideal” student, with his realism indices and his realism graphs. 
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Figure 6: Student 11 *s “spectrum of performances” 



Score (PRE) : 38.61 % 
Score (POST) : 60.32 % 


Realism (PRE) : 0.9 (verv good) 
Realism (POST) : 0.88 (very good) 


Absolute gain : 21,71 % 
Relative gain : 36,87 % 


Skewn. (PRE) : Incorr. A. : 0,73 Corr. A. : -1,42 
Skewn. (POST) : Incorr. A. : 0,05 Corr. A. : -3,1 



Table 7: Student ll’s “performances card” 
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Realism (pretest) 



Realism (post-test) 



Figure 8: Student ll’s “realism graphs” 

We defined an « ideal student » as a student who makes progress in the two expected levels : 



(1) in his knowledge of English : The curve of his post-test ’’goes up” towards the right (peak at C5). His 
relative gain is high (37%). 

(2) in realism : In this case, St. 11 does not increase his realism index at the post-test, but this index was 
already ’’very good” at the pretest, refering to [Gilles 96] norms. 

Interviewed, St. 1 1 declared he appreciated the self-directed learning session. The English level suited him very 
well. He did not have the opportunity to consult his pretest before the learning session. Nevertheless, his 
process of navigation in the CD-Rom was efficient, seeing that the rate (55%) of his “seized opportunities to 
learn” is higher than the mean of the group’s rate (45%). 



Conclusions 

There does not exist something such as “the only one good way to explore a hypermedia”. Moreover, students 
are characterised by what [Denis & Leclercq 94] called “ mathetical ambivalence ”, i.e. the need to rapidly 
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change from an autonomous learning situation (let me try, let me search, let me create) to a dependent situation 
(tell me, show me, correct me) and vice versa. The hypermedia “L’anglais d’aujourd’hui en 90 lemons ” allows 
these reversible shifts. 

One kind of problems encountered by a hypermedia user concerns the navigation in the great mass of 
information. In collaboration with the CNRS - IRPEACS of Lyon, we are developing and experiencing new 
tools to facilitate the navigation in a hypertext [Zeiliger et al. 96 & 97]. 

The analysis of the first results stressed interesting recommendations to carry on with the research into this 
field. The second part of our research project will aim to compare the impact of self-directed learning on the 
one hand and of collaborative learning in the other hand of university students navigating in hypermedia. 
That’s why, this year, 30 students were asked to experiment the two situations in English learning. Different 
indices such as the evolution of their “ spectrum of performances ”, their (absolute and relative) gains on 
competencies, their satisfaction,... will be analysed and compared according to the two experimental 
conditions. 

It is hoped that from this research recommendations could be drawn about learning with hypermedia to attain 
objectives that the learner (or the learners’ group) has assigned to himself. 
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